Defining an Agent List

You can assign jobs to run using Airflow agent/adapter lists similar to the way you assign jobs to individual agents. With Airflow lists, however, you have access to useful functions not available with individual connections.

You can

  • Specify alternate connection(s) to run your jobs if the primary connection is unavailable.

  • Balance the workload of many jobs among the process servers in the list.

  • Broadcast jobs to run on all the servers in the list at the same time.

To define an Airflow Agent list:

  1. Click Definitions > Agent Lists > Airflow on the Navigation pane to display the Airflow Agent Lists pane.

  2. Right-click the Airflow Agent Lists pane and click Add Agent List from the shortcut menu.

    OR

    Choose Add on the TA toolbar to display the Agent List Definition dialog.

  3. Enter the name of the Adapter list (up to 60 characters) in the List Name field.

  4. Click the Parent List field. You can choose a parent list if you want to create nested lists. If you specify a parent list, the list you are editing is a child list of the parent and will appear below its parent in the list hierarchy.

  5. Click the List Type list and choose the type of list based on your preference for jobs to be assigned to an Airflow connection in the list.

    • Balanced Available – TA chooses a connection from the Agents Selected section based on lightest load when the job is ready to launch with an open slot. The Balanced Available list uses Airflow connection load information to decide which connection in the list runs a job when its dependencies are met. You can choose this option to balance your workload across all the available connections in the list.

    • Balanced Load – TA chooses a connection from the Agents Selected section based on lightest load when the job is ready to launch, even if a slot is not available. The Balanced Load list uses Airflow connection load information to decide which connection in the list runs a job when its dependencies are met. You can choose this option to balance your workload across all the available connections in the list.

    • Broadcast – TA runs the job on every connection in the Agents Selected section. For example, you can use this option to schedule a maintenance process on each machine in the list using the same job.

    • Ordered – In support of high availability, TA chooses the first available agent to run the job based on the order that the connections appear in the Agents Selected section. For example, if the first (primary) connection in the list is not available, TA tries the second (alternative) connection in the list. You might want to use this list type for critical jobs where multiple connections are capable of processing the request.

    • Random – Chooses connections from the Agents Selected field randomly. This is a basic form of workload balancing. For more advanced load balancing, choose the Balanced option.

    • Rotation – TA cycles through the list of connections in the Agents Selected field and launches jobs assigned to that list in rotation. When the end of the list is reached, the first connection in this list is chosen. This is a form of workload balancing where you can predict where the next job will run.

  6. Highlight the appropriate Airflow agent from the Agents Available section on the List tab.

  7. Click the left arrow to add the selected connection to the list by moving it from the Agents Available section to the Agents Selected section. You can also drag and drop connections from one section to another.

  8. Click OK.